The present invention relates to digital video, and more particularly relates to techniques for recognizing digital video so that certain usage-control systems may be implemented.
Certain applications would benefit from knowing whether a media-specific watermark is present in, effectively, a xe2x80x9cbucket of bits.xe2x80x9d In other words, one only knows, a priori, that one has a set of binary information, and even the very existence of a media objectxe2x80x94whether audio, video or still imagexe2x80x94is in question. It would be nice to first answer the question of whether a media object exists within this bucket of bits, followed by determining the specific form or format of that object (e.g., MPEG stream, JPEG image, BMP image, WAV file, etc.). Once the media object(if any) is characterized, it can be examined for a watermark. This disclosure provides a method for doing just this. It is presented in the harshest case of complete ignorance about the bucket of bits, whereas most practical applications will have additional knowledge about the data that can assist in correctly identifying its form and function, thus assisting in the watermark reading process.
Imagine we start with 10 Kilobytes of data randomly taken from any of the dozens of the digital physical media available (CD""s, DVD""s, Hard drives, memory, communication streams . . . anywhere at all). These bytes just show up on your doorstep somehow. You ask, is this a media object I should be interested in? And if so, is there a watermark in there?
Let""s begin with two extremes, one where it is generally impossible to figure out whether or not a media object is present and what its form is, and one where it is relatively straightforward. We will then move on to the gray area in between.
The xe2x80x9cimpossiblexe2x80x9d case is where some data object such as text, a program, a video, an image, whatever, has been encrypted using decent modern cryptographic methods. The essential point of cryptography is to not let some random person know what is xe2x80x9cwithin the data.xe2x80x9d Generally speaking, gross level characteristics and statistics of such data have been tidily swept clean and no discernable patterns can be found. The act of determining whether a watermark exists in such data is predicated upon specific breakdowns in cryptographic procedure, an area we will happily steer around in this disclosure and leave for braver souls.
On the other hand, one example of very many straightforward cases might be the following. A program is written simply to hunt for JPEG headers. Byte by byte, it simply walks through the whole bucket of bits and, bit by bit or byte by byte, simply xe2x80x9cassumesxe2x80x9d that a given bit or byte is the beginning of a JPEG header and sees if it can coherently decode the header using that bit/byte as a starting point, not quite knowing exactly where the ending point is. (Some headers have very precise lengths, speaking to this latter point about not knowing how long it is). As with the millions of monkeys typing on their keyboards eventually producing Hamlet, every now and then this program will think it has found a valid header when it has indeed found garbage, but it can then either begin to look for telltale huffman streams and refine its ability to recognize a JPEG image, or, more simply, it can simply not care it might be finding garbage and invoke a watermark reading process on xe2x80x9cpresumedxe2x80x9d JPEG data. The worst that can happen is that no watermark is found or read.
(The present technology is applicable with any digital watermarking technique; exemplary watermarking techniques are disclosed in U.S. Pat. Nos. 5,832,119, 5,825,892, and 5,915,027.)
In this simpler case of looking for a JPEG header, the process of reading a watermark from this initially unknown bucket of bits is then to simply run the JPEG header-hunter routine followed by any standard watermark reading routine. End of story.
Let""s now look at some more difficult cases and attempt to specify certain operations that could underpin any generalized watermark reader on generalized unknown data.
Let""s look at the case of an MPEG stream where it may be impractical to xe2x80x9csearch for telltale headers.xe2x80x9d It might even be that a non-crytpographic-grade scrambling function has been applied to the pure MPEG stream, i.e., a scrambling function which is meant to stop average consumers from co-opting materials but certainly has no pretensions of stopping cryptographic pro""s.
In the non-scrambled version, a xe2x80x9crandom sampling watermark detectorxe2x80x9d can be built which basically operates on some small but non-trivial xe2x80x9cchunkxe2x80x9d of data, essentially asking a series of questions all revolving around the basic question of xe2x80x9cdo you have MPEG-like data structures.xe2x80x9d Examples include: xe2x80x9care there data structures which resemble sequences of macroblocks, with subsequences of 4 blocks within the macroblocks.xe2x80x9d More specifically, and similar to the JPEG header-hunter, the existence of macroblocks can be assumed, but with an unknown xe2x80x9cstart point;xe2x80x9d many trial start points can be tried and some start points (followed by typical decoding steps associated with MPEG/Huffman decoding) will start to produce what appear to be true macroblock data. Options include assuming it is indeed macroblock data and attempting watermark reading, but more likely other questions will be asked of the data before the step of watermark reading is invoked.
Returning back to the xe2x80x9cdo you have MPEG-like data structuresxe2x80x9d issue then, another example might include looking for interframe sub-header data structures as well, or looking for these only if an earlier step thinks it might be seeing true macroblock data. One can also treat raw ones and zeros of the data stream as a time waveform and look at is spectral and statistical properties and look for xe2x80x9cMPEG class signaturesxe2x80x9d within these transformed spaces.
In any event, once some reasonable level of confidence is obtained that an MPEG object exists within the bucket of bits and once some general notion of where macroblocks are synchronized, a watermark reading routine can be invoked and a xe2x80x9cpositive readxe2x80x9d (in the case where false positives are pre-designed to be very unlikely) is the best way of making sure that an object is an MPEG object.
One final note on the non-scrambled MPEG case, the xe2x80x9cdo you have MPEG-like data structuresxe2x80x9d questions can be seen as distinct modules, each with a probability attached. These modules can be stacked in order to gain sufficient confidence that an MPEG stream is present. Furthermore, interim states of confidence can invoke grabbing of more data and reapplying the question modules to more data if a sufficient level of confidence is not reached right away. The issue here gets down to xe2x80x9cwhich is computationally cheaper,xe2x80x9d getting more data and building confidence, or simply going for a full watermark read routine.
Moving on to the scrambled MPEG video case, a similar philosophy of asking a set of modularized questions can be used, only they will be much more along the classic cryptographic analysis lines than in the non-scrambled case which can search for specific structures. Furthermore, presumably most applications which intend to look for these scrambled signals will also have the means to unscramble the data if a scrambled stream is indeed found. Building in the means to xe2x80x9csleuth the scramblingxe2x80x9d is certainly not an intent of this disclosure. Implicit in all of this is that the process to xe2x80x9cdescramble the signal using all arbitrary starting pointsxe2x80x9d is computationally more expensive than the process to ask the questions xe2x80x9cdo you have scrambled MPEG-like data characteristics.xe2x80x9d Key words have changed in this phrasing of the question.
In the scrambled case, then, we find a three step process: 1) look for scrambled MPEG signal characteristics, including looking for specific commercial flavors of scrambling; 2) descramble the data if a certain confidence level is reached in number 1, where one may need to grab more data if the initial chunk grabbed is insufficient for descrambling; then 3) invoke any given watermark reading routines properly synchronized to the macroblocks (which is assumed to be feasible due to the proper descrambling method being used in step 2).
Generalizing all of this to arbitrary media objects, not just JPEG and MPEG, any given target media type would have their question modules with procedures tuned to characteristics and structures unique to their specific type/format. Question modules (sometimes just one) are stacked and applied to the bucket of bits. Confidence levels either remain low or begin to peak when a given media object being searched for starts showing up in the bucket of bits. Then watermark reading routines are invoked, routines associated with the given media object that is showing up on the confidence radar screen.
Generalizing this yet further, these techniques can be applied to the time waveform of data into and out from a CPU or any digital node within a digital system. The question modules, the decoding (descrambling) modules, and the watermark types can all be tuned to a given node, and a generalized watermark reader can effectively watch digital traffic go by, then potentially add value to consumers who want to enable certain operations connected to certain watermarked media objects, or can add value to copyright holders who have every intent to retain their rights to their material and ensure that any uses of their material have been properly paid for or permitted.
One particular application of such technology is as an adjunct to a copy protection system for digital video. A variety of such systems are known, as illustrated by U.S. Pat. Nos. 5,915,018, 5,949,877, 5,923,754, and 5,883,958, and by the Content Scrambling System (CSS), licensed by Matsushita. Video distributed in accordance with the CSS standard is encrypted for storage on a DVD disk. Playback devices encountering such CSS-encrypted data must have CSS decryption hardware or software for the playback to proceed. Matsushita is working to incorporate a digital watermark component to provide further playback and copy control features. Two industry consortia, one composed of the present assignee, Macrovision and NV Philips, and the other composed of IBM, NEC, Hitachi, Sony and Pioneer, have proposed specific watermarking systems that may be suitable. (See www.dvcc.com/warp for specifics of the proposals.) For expository convenience, such watermarking systems are here called CSS watermarking systems.
A problem arises, however, if a hacker gains access to the un-encrypted video, e.g., by intercepting the analog drive signals fed to the display device. Such video signals can be re-digitized, and stored on writable media (the writable DVD is not far off) for playback without CSS constraints. Such pirated video clearly poses a large economic threat to Hollywood and other content providers. However, such video would still include the CSS watermark and, if the non-CSS-encoded data being read from a DVD disk can be recognized as MPEG video, a watermark detection operation can be invoked.
Desirably, then, generic DVD drives are provided with built-in hardware or firmware that checks the raw data read from the disk for MPEG-like characteristics. (The details of prior art DVD drives are familiar to the artisan and are not belabored here.) If suspected MPEG data is identified, the hardware and/or firmware invokes a watermark reading operation (which may also be integrated with the drive, or external to it, and may be implemented in hardware, firmware, or software) to see if the data carries a CSS watermark. (This operation may, or may not, require full or partial decoding of the MPEG data stream. A wide variety of techniques for detecting watermarks from MPEG data streams are known.) If such a watermark is detected, the hardware and/or firmware can simply interrupt the read operation, or the provision of data on the output bus; no CSS watermarked video should be encountered on a DVD that is not CSS-encrypted.
It will be recognized that it is not always necessary to read (decode) the watermark. Once the presence of a CSS watermark has been detected, its data payload is substantially irrelevant. Often, implementation can be simplified (e.g., gate count can be reduced, or MIPs processing burden can be be lessened) by skipping the final decoding steps once the watermark""s presence has been ascertained.
To avoid certain hacks, the MPEG-detection system may be arranged to work on out-of-order sectors of data, in case the disk is read non-sequentially. Similarly, the system may be arranged to check for simple data inversion in its screening of the DVD data for possible MPEG video.
FIG. 1 shows such a system 10 that includes a sensor 14 for reading data from a DVD disc 12, an MPEG screening system 16, and a watermark sensor 18 (e.g., watermark reader) responsive to the output of the MPEG screening system. Desirably, these components 14-18 are provided as part of a DVD drive unit.
FIG. 2 shows a method 20 that may be performed by the system 10 of FIG. 1. Data is read from the storage medium (22) and is examined for one or more attributes associated with MPEG video (24). If such attribute is found (26), the process next determines whether a watermark is present (28). If so (30), the system responds by preventing playback (32).
To provide a comprehensive disclosure without unduly lengthening this specification, the above-cited patents are incorporated by reference.